Region-based processing of predicted pixels

ABSTRACT

A method for decoding a compressed video data sequence containing one or more coded pixel blocks. The compressed video sequence is buffered. Prediction information for each of the coded pixel blocks is reviewed. One or more groups of coded pixel blocks are formed based on the reviewed prediction information such that the coded pixel blocks within a given group have similar prediction dependencies and/or at least do not depend on a reconstructed pixel within a group of received pixel blocks to enable parallel decoding. The formed groups are scheduled for processing and subsequently decoded to produce a decoded video data sequence.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 60/737,772, filed Nov. 18, 2005, herein incorporated by reference inits entirety.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention generally relates to video encoders. Morespecifically, the present invention provides the grouping of coded pixelblocks having similar prediction dependencies to more quickly decode acompressed video data sequence.

Background Art

FIG. 1 is a functional block diagram of an encoder-decoder system 100.The encoder-decoder system 100 includes an encoder 102 and a decoder104. The encoder 102 receives source video data from a video source 106.The encoder 102 codes the source video data into a compressed bit streamfor transmission to the decoder 104 over a communication channel 105.The communication channel 108 can be a real-time delivery system such asa communication network (e.g., a wireless communication network) or acomputer network (e.g., the Internet). Alternatively, the communicationchannel 106 can be a storage medium (e.g., an electrical, optical ormagnetic storage device) that can be physically distributed. Overall,the topology, architecture and protocol governing operation of thecommunication channel 108 is immaterial to the present discussion unlessspecifically identified herein.

The decoder 104 receives and decodes the compressed bit stream toreproduce the source video data. The decoder 104 can then provide thereproduced source video data to a video display device 110. FIG. 1 showsa single decoder 104 but is not limited as such. That is replicas orcopies of the compressed bit stream can be provided to multiple decoderslocated at different locations. In this way, the source video data canbe encoded once and distributed to the decoders for decoding atdifferent times as is conventional or well known in the art.

The encoder 102 and the decoder 104 can be implemented in hardware,software or some combination thereof. For example, the encoder 102and/or the decoder 104 can be implemented using a computer system. FIG.2A is a simplified functional block diagram of a computer system 200.The computer system 200 can be used to implement the encoder 102 or thedecoder 104 depicted in FIG. 1.

As shown in FIG. 2A, the computer system 200 includes a processor 202, amemory system 204 and one or more input/output (I/O) devices 206 incommunication by a communication ‘fabric.’ The communication fabric canbe implemented in a variety of ways and may include one or more computerbuses 208, 210 and/or bridge devices 212 as shown in FIG. 2A. The I/Odevices 206 can include network adapters and/or mass storage devicesfrom which the computer system 200 can receive compressed video data fordecoding by the processor 202 when the computer system 200 operates as adecoder. Alternatively, the computer system 200 can receive source videodata for encoding by the processor 203 when the computer system 200operates as an encoder.

The computer system 200 can implement a variety of video codingprotocols such as, for example, any one of the Moving Picture ExpertsGroup (MPEG) standards (e.g., MPEG-1, MPEG-2, or MPEG-4) and/or theInternational Telecommunication Union (ITU) H.264 standard. Most codingstandards are designed to operate across a variety of computingplatforms. Accordingly, many coding standards find application infeature rich computing devices (e.g. personal computers or gamingdevices) and also in feature poor computing devices (e.g., singledigital signal processing (DSP) devices).

To accommodate the broad variety of computing devices, most codingstandards are designed with unsophisticated computer systems in mind.Specifically, many coding standards are designed to be implemented thesame way on a feature rich system as they are on a feature poor system.Feature poor systems typically have limited memory and processorcapabilities. Accordingly, due to the design of many coding standards,the improved memory and processor capabilities of a feature rich systemare not fully exploited. For example, during the coding of an Intracoded frame (I-frame), macroblock data may be coded with reference toother macroblocks in the same frame. A prediction reference (e.g., aprediction vector) can be generated to specify a location from withinpreviously coded macroblocks from which a prediction will be made. Thistechnique can create a long chain of predictions, requiring macroblocksto be retrieved from memory and then decoded in a serial fashion (e.g.one macroblock at a time).

FIG. 2B illustrates the contents of the memory system 204 depicted inFIG. 2A. As shown in FIG. 2B, the memory system 204 can include codedvideo data 220, decoded reference frames 222 and a currently decodedframe 224 that is partially decoded. The decoded reference frames 222can include previous reference frames 226 and future reference frames228. Previous reference frames 226 are frames that occur earlier in timethan the current frame being decoded by the processor 202. Futurereference frames 228 are frames that occur later in time than thecurrent frame being decoded by the processor 202. A currently decodedframe 224 can depend on previous reference frames 226 and/or futurereference frames 228.

In feature poor computing devices, the decoded reference frames 222needed by the processor 202 to decode a current frame 224 may takeseveral memory cycles to retrieve since the bandwidth of a memory system204 in a feature poor computing device is generally limited. Further,due to the serial nature of decoding the coded data 220, different setsof decoded reference frames 222 may be retrieved for each coded videoframe 220. This can result in the same decoded reference frames 222being retrieved several times as the coded video data 220 is decoded.

As previously mentioned, feature rich computing devices typicallypossess much greater memory capacity and memory bandwidth in comparisonto feature poor devices. Accordingly, these devices are able to access,retrieve and process data in quantities much larger than a singlemacroblock at a time. Theoretically, the performance of a feature richcomputing device implementing a coding standard is much greater thanwhat is currently achieved. Performance is limited due to the serialnature of decoding. In particular, performance is mainly limited by (a)the processor being idle when macroblock data is being read in from amemory device or I/O device because the processor needs prediction datacontained in data yet to be retrieved; and (b) the memory device or I/Odevice being idle because the processor is busy decoding data thatrequires large amounts of processing.

Accordingly, what is needed is a processing system capable of exploitingthe improved processor and memory capabilities of feature rich computingdevices to more quickly decode video data compressed according toconventional coding techniques. In particular, the processing systemshould be capable of reading and decoding multiple chunks of video data(e.g., multiple macroblocks or multiple blocks of pixels) wheneverpossible.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate the present invention and, togetherwith the description, further serve to explain the principles of theinvention and to enable one skilled the pertinent art to make and usethe invention.

FIG. 1 is a functional block diagram of an encoder-decoder system.

FIG. 2A is a simplified functional block diagram of a computer system.

FIG. 2B illustrates the contents of a memory system depicted in FIG. 2A.

FIG. 3 provides a flowchart illustrating a method for grouping pixelblocks together so that the group can be processed together more quicklyaccording to an aspect of the present invention.

FIG. 4 depicts a frame of video data to illustrate data decodingaccording to an aspect of the present invention.

FIG. 5 illustrates a portion of a coded video sequence generatedaccording to an aspect of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention provide systems, apparatuses andmethods whereby groups of coded pixel blocks having similar predictiondependencies are defined and efficiently scheduled for decoding. In thisregard, the present invention enables a compressed video data sequenceto be decoded in less time and with less power by leveraging improvedprocessor and memory capabilities. A group of coded pixel blocks havingsimilar prediction dependencies is understood to mean coded pixel blocksthat can share a motion vector or reference frame or that do not dependon a reconstructed pixel block within the group.

FIG. 3 provides a flowchart 300 illustrating a method for grouping pixelblocks together (e.g., grouping pixel blocks together that have similarprediction dependencies) so that the group can be processed togethermore quickly, according to an aspect of the present invention.

At step 302, a processor of a decoder can buffer channel data in amemory. The channel data can be compressed video data. The processor andmemory can be components of a computer system operating as decoder. Theimplemented decoder can operate according to or be operable with variousvideo compression standards such as, for example, any one of the MovingPicture Experts Group (MPEG) standards (e.g., MPEG-1, MPEG-2, or MPEG-4)and/or the International Telecommunication Union (ITU) H.264 standard.Overall, the compression scheme used by the decoder is immaterial to thepresent discussion unless specifically identified herein.

At step 304, the processor begins a new group of pixel blocks (e.g.,macroblocks) at an arbitrary location and examines the pixel blockswithin the buffer. Typically, a new group of pixel blocks beginsimmediately after the previous group.

At step 306, a pixel block can be retrieved. The current pixel block canbe considered the ‘i^(th)’ pixel block.

At step 308, the coding parameters of the i^(th) pixel block can beexamined to determine if the current pixel block can be decoded inparallel with the other pixel blocks in the group. If the current pixelblock does not depend on a reconstructed pixel within the group, thenthe current pixel block can be added to the current group at step 310.Steps 306 and 308 can then be repeated for additional pixel blocksremaining in the buffer. If the current pixel block does depend on areconstructed pixel within the group, then the group can be closed atstep 312 without including the current pixel block. If the current pixelblock does depend on another pixel block within the group (i.e., areconstructed pixel with the group), then the current pixel block can beconsidered a non-qualifying pixel block.

Further, at step 312, the group formed can be scheduled for decoding inthe manner most efficient for the processor. That is, one or more groupsof pixel blocks generated according to the previous steps can beretrieved or read in bulk. Thereafter, the one or more groups of pixelblocks can be processed as additional channel data is fed into thebuffer since the one or more groups of pixel blocks do not depend on orrequire data yet to be retrieved. In this way, the processor and memorysystem can operate simultaneously, thereby decreasing the time needed todecode a compressed video stream.

In general, the generated groups of coded pixel blocks can be retrievedin any manner. That is, each bulk read operation can include only aportion of a single group (e.g., if a given group is large) or multiplegroups (e.g., if some of the included groups are small). This allows anygiven group of coded pixel block to be retrieved and processed in achunk size that may be larger, smaller or equal to the group size.

Various groupings of pixel blocks are possible using the methodillustrated by the flowchart 300. The groupings can be made on atemporal and/or spatial basis. For example, a group can compriseadjacent intra-frame pixel blocks that can be predicted (and thereforedecoded) without using decoded values within the group. A group can alsocomprise adjacent inter-frame pixel blocks that can be predicted withoutusing decoded values within the group. In this way, it is possible togroup together pixel blocks that use multiple reference frames and/ormultiple motion vectors. Further, a group can comprise adjacentinter-frame pixel blocks that share one reference frame and/or onemotion vector. These grouping decisions can be made, for example, atstep 308 using information on the current pixel block retrieved at step305.

FIG. 2A, representing a feature rich computing device, can be used toimplement an embodiment of the present invention. The computer system200 can read from any memory system or location (e.g., a hard drive,cache, etc) when decoding data. Further, these retrieval operations canresult in the reading of large amounts of information at a time and isnot limited to retrieving or reading a single pixel blocks at a time.Accordingly, an encoder of the present invention can retrieve requireddecode reference frames 222 in bulk in order to process a group of pixelblocks. For example, all of the decoded reference frames 222 needed todecoded a scheduled group of coded pixel blocks can be retrieved in onememory cycle so as to decrease the time to decode the group of pixelblocks. The ability to more quickly gather all of the information neededfor decoding, and to decode efficiently in groups, enables an encoder ofthe present invention to reduce a time needed to decode a compressedvideo sequence.

FIG. 4 depicts a frame of video data 400 to illustrate data decodingaccording to an aspect of the present invention. The frame 400 isdivided into a plurality of pixel blocks 402. For each pixel blockshown, there can be a coded representation of the pixel block stored ina decode buffer. A decoder of the present invention can process thepixel blocks in some kind of order. For example, the decoder can processthe pixel blocks serially (e.g., raster scan facsimile).

In FIG. 4, the pixel block group 404 (i.e., those pixel blocks 402containing an ‘X’) is depicted as having been decoded. Specifically, thedecoder has already read and decoded the coded data representing thepixel block group 404. According to an aspect of the present invention,the decoder defines a new pixel block group 420. The new pixel blockgroup 420 can be defined by the decoder first examining the predictionreferences of each of the pixel blocks. If the prediction references ofan individual pixel block depend on the decoded pixel blocks 404, thepixel block can be added to the new group 420. The new group 420 can beclosed or end when it is determined that a currently examined pixelblock refers to another pixel block within the new group 420. The newlydefined group 420, and therefore the constituent pixel blocks of thegroup 420, can then be scheduled for decoding.

In an embodiment of the present invention, a bit stream syntax isintroduced to identify a group of pixel blocks having similar predictiondependencies. The bit stream syntax and groupings can be generated by anencoder of the present invention. FIG. 5 illustrates a portion of acoded video sequence 500 generated according to an aspect of the presentinvention. The portion of the coded video sequence 500 can include aheader 503 and a payload 504. The portion of the coded video sequence500 can comprise a portion of a bit stream representing the coded videosequence parsed into portions as shown in FIG. 5. The header 502 caninclude information regarding the contents of the payload 504. Thepayload 504 includes a number of pixel blocks (e.g. macroblocks) 5064through 506-N. The header 502 is shown prepended to the payload 504 inFIG. 5 but can alternatively be appended to the payload 504.

The pixel blocks 506-1 through 506-N can be grouped together accordingto prediction dependencies. The pixel blocks 5064 through 506-N can begrouped together in a number of ways. For example, the payload 504 canbe (a) a group of adjacent inter pixel blocks that share a common motionvector and/or reference frame; (b) a group of adjacent inter pixelblocks that can be predicted without using decoded values within thegroup; (c) a group of adjacent intra pixel blocks that can be predictedwithout using decoded values within the group; or (d) a group of pixelblocks that do not depend on any reconstructed pixel from the group ofpixel blocks. In this way, the present invention allows the formation ofa group of pixel blocks for decoding that do not share a motion vectoror a reference frame. The payload 502 can include a variety ofinformation on the characteristics of the payload 504 and/or itsconstituent components. For example, the payload 504 can includeinformation indicating the type of grouping carried by the payload 504as well as the number of pixel blocks 506 within the payload. A header502 can be associated with each payload 504 comprising the compressedbit stream and can be considered a group identifier of the payload 504.Further, the compressed bit stream can comprise payloads 504 ofdifferent types distributed throughout the compressed bit stream in anymanner.

An encoder of the present invention can generate the payload group 504and the header 502 as shown in FIG. 4. For example, after receiving andcoding an uncompressed video sequence to produce a coded video sequence,an encoder of the present invention can scan the coded video sequence toform multiple payload groups 504 and associated headers 502 to form aformatted coded video sequence. The formatted coded video sequence canthen be provided to a decoder of the present invention. The decoder canprocess and decode the formatted and coded video sequence quickly toreproduce the original uncompressed video sequence since the coded videosequence is provided in chunks having similar prediction dependenciesand/or at least do not depend on a reconstructed pixel within a group ofreceived pixel blocks. Accordingly, the decoder can more efficientlyschedule retrieval and processing of the coded and formatted videosequence stored, for example, in an associated decoder buffer.

As a two-step process, generation of a formatted coded video sequencecan be used to optimize the original coding of the video sequence. Thegeneration of a formatted coded video sequence can also be producedusing a one-step process whereby an encoder of the present invention canencode and format a video sequence in a serial fashion. An encoder ofthe present invention may also adjust the coding and formatting processbased on the capabilities of remote decoder that may receive the codedand formatted video data sequence. Further, an encoder of the presentinvention may be capable of exploiting one or more side channels to sendinformation (e.g., user-defined data and/or encoding/formatting data) toa decoder. In this way, an encoder can provide a decoder with a codedvideo sequence formatted in such a way (with or without supplementalinformation) that it can be decoded more easily and more quickly.

CONCLUSION

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample and not limitation. It will be apparent to one skilled in thepertinent art that various changes in form and detail can be madetherein without departing from the spirit and scope of the invention.Therefore, the present invention should only be defined in accordancewith the following claims and their equivalents.

What is claimed is:
 1. A method for decoding a compressed video datasequence, comprising: buffering the compressed video sequence, thecompressed video sequence comprising one or more coded pixel blocks;reviewing prediction information of the buffered coded pixel blocks;organizing coded pixel blocks into groups based on the reviewedprediction information, wherein coded pixel blocks that are assigned toa common group have prediction references that refer to image contentoutside of the respective common group; scheduling decoding of the codedpixel blocks on a group-by-group basis; and decoding the groups of codedpixel blocks according to the group-by-group schedule to produce adecoded video data sequence.
 2. The method of claim 1, wherein theprediction information comprises a motion vector.
 3. The method of claim1, wherein the prediction information comprises a reference frame. 4.The method of claim 1, wherein the organizing comprises forming at leastone common group comprising coded pixel blocks that can be predictedindependently of one another.
 5. The method of claim 4, wherein thecoded pixel blocks of the at least one group are intra-frame pixelblocks.
 6. The method of claim 4, wherein the coded pixel blocks of theat least one group are inter-frame pixel blocks.
 7. The method of claim1, wherein the organizing comprises forming at least one group havingcoded pixel blocks that share one motion vector and one reference frame.8. A computer-readable memory having stored thereon a plurality ofinstructions for decoding a compressed video sequence comprising one ormore coded pixel blocks, which when executed by a processor, cause theprocessor to: retrieve coding parameters of a coded pixel block; comparethe coding parameters of the coded pixel block to coding parameters of acurrent group of pixel blocks; add the coded pixel block to the currentgroup of pixel blocks if the coding parameters of the coded pixel blockrefer to image content outside of the current group of pixel blocks; andschedule decoding of the current group of pixel blocks once the currentgroup of pixel blocks is closed, wherein the current group of pixelblocks is closed upon detection of a first non-qualifying coded pixelblock that refers to image content inside the current group of pixelblocks.
 9. The computer-readable memory of claim 8, further comprisingcausing the processor to decode a chunk of the current group of pixelblocks.
 10. The computer-readable memory of claim 8, wherein the codingparameters of the coded pixel block refer to image content outside ofthe current group of pixel blocks when the coded pixel block can bepredicted independently of other coded pixel blocks within the currentgroup of pixel blocks.
 11. The computer-readable memory of claim 10,wherein the current group of pixel blocks comprises one or moreinter-frame pixel blocks.
 12. The computer-readable memory of claim 10,wherein the current group of pixel blocks comprises one or moreintra-frame pixel blocks.
 13. A method for encoding a video datasequence, comprising: coding the video data sequence to generate a codedvideo sequence comprising one or more coded pixel blocks; reviewingprediction information for each of the one or more coded pixel blocks;forming one or more groups of coded pixel blocks based on the reviewedprediction information, wherein coded pixel blocks that are assigned toa common group have prediction references that refer to image contentoutside of the respective common group; and associating a groupidentifier with each common group of coded pixel blocks to form aformatted coded video data sequence.
 14. The method of claim 13 whereincoding comprises coding the video data sequence according to any one ofthe Moving Picture Experts Group (MPEG) standards.
 15. The method ofclaim 13 wherein coding comprises coding the video data sequenceaccording to an International Telecommunication Union (ITU) H.264standard.
 16. The method of claim 13, wherein the forming comprisesforming at least one common group having coded pixel blocks that can bepredicted independently of one another.
 17. The method of claim 16,wherein the coded pixel blocks of the at least one common group areintra frame pixel blocks.
 18. The method of claim 16, wherein each groupidentifier specifies a number of coded pixel blocks contained within anassociated common group of coded pixel blocks.
 19. The method of claim13, wherein the coded pixel blocks of the at least one common group areinter frame pixel blocks.
 20. A method for decoding a compressed videodata sequence, comprising: buffering the compressed video sequence, thecompressed video sequence comprising one or more coded pixel blocks;reviewing prediction information of the buffered coded pixel blocks;organizing coded pixel blocks into groups based on the reviewedprediction information, wherein coded pixel blocks that are assigned toa common group refer to image content data outside the respective commongroup; scheduling decoding of the coded pixel blocks on a group-by-groupbasis; and decoding the groups of coded pixel blocks according to thegroup-by-group schedule to produce a decoded video data sequence.
 21. Amethod for encoding a video data sequence, comprising: coding the videodata sequence to generate a coded video sequence comprising one or morecoded pixel blocks; reviewing prediction information for each of the oneor more coded pixel blocks; organizing one or more groups of coded pixelblocks into groups based on the reviewed prediction information, whereincoded pixel blocks that are assigned to a common group refer to imagecontent outside the respective common group; and associating a groupidentifier with each common group of coded pixel blocks to form aformatted coded video data sequence.